Practical 3: Web Traffic
Colin Steffe, Virany Kho, Jasmine Mawjee, Adela Sahraoui
11/25/2021
Informative Plots
Time Series
If we see a trend, the time series is not stationary (i.e. does not depend on the time of the observation).
Seasonal plot
If we see a seasonality, the time series is not stationary (i.e. does not depend on the time of the observation).
Elizabeth_II
Je sais pas pk ca tourne pdt mille ans sans voir le résultat
- time series decompostion
Autocorrelation plot
Autocorrelation quantifies the relationship between lagged values of a time series.
Modified Series
\[ R_i = \frac{X_i - X_{i-7}}{X_{i-7}} * 100 \] \(R_i\) is the modified time series where each series (above \(R_8\)) of pageviews is the relative percentage change time series between one observation at time \(i\) and the seventh before \(i-7\) ?
2016_Summer_Olympics
To compare the time series before and after modification
Peaks-Over-Threshold
- Examine the suitability for each time series \(R_i\) separately
- Explain your choice of threshold and briefly comment on the diagnostic plots for each model
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
## $y
## [1] "Latitude of Seismic Events"
##
## attr(,"class")
## [1] "labels"
Suitability of POT: - Princess Margaret, United Kingdom, United States (?), Wiston Churchill do not seem to be suitable for POT because of the too few numbers of exceedances.
- Estimate the 99%-quantile of each series and give a corresponding measure of uncertainty = provide an interval of values within which the true value of the threshol is believed to lie with a stated probability (MSE ? with confidence interval ?)
library(evd)
# data frame for the 99 quantile and measure of uncertainty for all the type
data99 <- data.frame(matrix(0, nrow = 2, ncol = length(unique(ts$type))))
colnames(data99) <- unique(ts$type)
for (i in 1:ncol(data99)){
# filter for the type
ts_type <- ts %>%
filter(type == names(data99)[i])
# remove na
ts_type <- ts_type %>%
filter(!is.na(`daily count modified`))
# compute 99 quantile
quantile99 <- quantile(ts_type$`daily count modified` , 0.99)
# save the quantile in the data.frame
data99[1,i] <- quantile99
# measure of uncertainty
# not sure about the mper argument
# doc of the function here : https://www.rdocumentation.org/packages/evd/versions/2.3-3/topics/fpot
uncertainty <- fpot(ts_type$`daily count modified`, threshold = thresholds[i], mper = quantile99)
# not sure if we need to save the r level or shape
data99[2,i] <- uncertainty$std.err[1]
}## Warning in fpot.quantile(x = x, threshold = threshold, start = start, npp =
## npp, : optimization may not have succeeded
## Warning in fpot.quantile(x = x, threshold = threshold, start = start, npp =
## npp, : optimization may not have succeeded
## Warning in fpot.quantile(x = x, threshold = threshold, start = start, npp =
## npp, : optimization may not have succeeded
## Warning in fpot.quantile(x = x, threshold = threshold, start = start, npp =
## npp, : optimization may not have succeededDetecting Simultaneous High Load
- Propose one graphical method
- Propose one numerical method
for detecting simultaneous high load across the 12 series provided, Which pages seem to have simultaneous high load?
library(extRemes)## Loading required package: Lmoments## Loading required package: distillery##
## Attaching package: 'extRemes'## The following objects are masked from 'package:evd':
##
## fbvpot, mrlplot## The following objects are masked from 'package:stats':
##
## qqnorm, qqplot# idea for graphical representation : block maxima by week colored by type
# https://rdrr.io/cran/extRemes/man/blockmaxxer.html
tsnona <- ts%>% filter(!is.na(`daily count modified`))
# compute block maxima
bm <- blockmaxxer(tsnona, blocks = tsnona$date, which="daily count modified")
library(plotly)
c <-ggplot(tsnona, aes(x=date, y=`daily count modified`)) + geom_point(aes(color = type)) + geom_point(data=bm,aes(date,`daily count modified`, fill = type), colour = "lightpink1")
ggplotly(c)# numerical method
# GDP model ?
#https://rdrr.io/cran/evir/man/gpd.html
library(evir)##
## Attaching package: 'evir'## The following object is masked from 'package:extRemes':
##
## decluster## The following objects are masked from 'package:evd':
##
## dgev, dgpd, pgev, pgpd, qgev, qgpd, rgev, rgpd## The following object is masked from 'package:ggplot2':
##
## qplotmodified_NoNA <- modified_ts %>% filter(!is.na(`daily count modified`))
gpd.model <- gpd(modified_NoNA$`daily count modified`, threshold = mean(thresholds))
gpd.plot <- tailplot(gpd.model)
#gpd.sf <- gpd.sfall(gpd.plot,0.99)library(FRAPO)## Loading required package: cccp## Loading required package: Rglpk## Loading required package: slam## Using the GLPK callable library version 4.47## Loading required package: timeSeries## Loading required package: timeDate## Financial Risk Modelling and Portfolio Optimisation with R (version 0.4-1)modified_pivot <- modified_NoNA %>% pivot_wider(values_from =`daily count modified`, names_from = type,date)
cov.mat <- cov(modified_pivot[,-1])
tdc(cov.mat, method = "EVT")## 2016_Summer_Olympics
## 2016_Summer_Olympics 1.0000000
## Diana,_Princess_of_Wales 0.3333333
## Elizabeth_II 0.3333333
## George_VI 0.3333333
## Prince_Philip,_Duke_of_Edinburgh 0.3333333
## Princess_Margaret,_Countess_of_Snowdon 0.3333333
## Queen_Victoria 0.0000000
## United_Kingdom 0.6666667
## United_States 0.6666667
## Winston_Churchill 0.3333333
## World_War_I 0.0000000
## World_War_II 0.3333333
## Diana,_Princess_of_Wales Elizabeth_II
## 2016_Summer_Olympics 0.3333333 0.3333333
## Diana,_Princess_of_Wales 1.0000000 0.6666667
## Elizabeth_II 0.6666667 1.0000000
## George_VI 0.6666667 1.0000000
## Prince_Philip,_Duke_of_Edinburgh 0.6666667 1.0000000
## Princess_Margaret,_Countess_of_Snowdon 0.6666667 1.0000000
## Queen_Victoria 0.3333333 0.6666667
## United_Kingdom 0.3333333 0.6666667
## United_States 0.3333333 0.6666667
## Winston_Churchill 0.6666667 1.0000000
## World_War_I 0.3333333 0.3333333
## World_War_II 0.6666667 0.3333333
## George_VI
## 2016_Summer_Olympics 0.3333333
## Diana,_Princess_of_Wales 0.6666667
## Elizabeth_II 1.0000000
## George_VI 1.0000000
## Prince_Philip,_Duke_of_Edinburgh 1.0000000
## Princess_Margaret,_Countess_of_Snowdon 1.0000000
## Queen_Victoria 0.6666667
## United_Kingdom 0.6666667
## United_States 0.6666667
## Winston_Churchill 1.0000000
## World_War_I 0.3333333
## World_War_II 0.3333333
## Prince_Philip,_Duke_of_Edinburgh
## 2016_Summer_Olympics 0.3333333
## Diana,_Princess_of_Wales 0.6666667
## Elizabeth_II 1.0000000
## George_VI 1.0000000
## Prince_Philip,_Duke_of_Edinburgh 1.0000000
## Princess_Margaret,_Countess_of_Snowdon 1.0000000
## Queen_Victoria 0.6666667
## United_Kingdom 0.6666667
## United_States 0.6666667
## Winston_Churchill 1.0000000
## World_War_I 0.3333333
## World_War_II 0.3333333
## Princess_Margaret,_Countess_of_Snowdon
## 2016_Summer_Olympics 0.3333333
## Diana,_Princess_of_Wales 0.6666667
## Elizabeth_II 1.0000000
## George_VI 1.0000000
## Prince_Philip,_Duke_of_Edinburgh 1.0000000
## Princess_Margaret,_Countess_of_Snowdon 1.0000000
## Queen_Victoria 0.6666667
## United_Kingdom 0.6666667
## United_States 0.6666667
## Winston_Churchill 1.0000000
## World_War_I 0.3333333
## World_War_II 0.3333333
## Queen_Victoria United_Kingdom
## 2016_Summer_Olympics 0.0000000 0.6666667
## Diana,_Princess_of_Wales 0.3333333 0.3333333
## Elizabeth_II 0.6666667 0.6666667
## George_VI 0.6666667 0.6666667
## Prince_Philip,_Duke_of_Edinburgh 0.6666667 0.6666667
## Princess_Margaret,_Countess_of_Snowdon 0.6666667 0.6666667
## Queen_Victoria 1.0000000 0.3333333
## United_Kingdom 0.3333333 1.0000000
## United_States 0.3333333 0.6666667
## Winston_Churchill 0.6666667 0.6666667
## World_War_I 0.3333333 0.0000000
## World_War_II 0.3333333 0.0000000
## United_States Winston_Churchill
## 2016_Summer_Olympics 0.6666667 0.3333333
## Diana,_Princess_of_Wales 0.3333333 0.6666667
## Elizabeth_II 0.6666667 1.0000000
## George_VI 0.6666667 1.0000000
## Prince_Philip,_Duke_of_Edinburgh 0.6666667 1.0000000
## Princess_Margaret,_Countess_of_Snowdon 0.6666667 1.0000000
## Queen_Victoria 0.3333333 0.6666667
## United_Kingdom 0.6666667 0.6666667
## United_States 1.0000000 0.6666667
## Winston_Churchill 0.6666667 1.0000000
## World_War_I 0.0000000 0.3333333
## World_War_II 0.3333333 0.3333333
## World_War_I World_War_II
## 2016_Summer_Olympics 0.0000000 0.3333333
## Diana,_Princess_of_Wales 0.3333333 0.6666667
## Elizabeth_II 0.3333333 0.3333333
## George_VI 0.3333333 0.3333333
## Prince_Philip,_Duke_of_Edinburgh 0.3333333 0.3333333
## Princess_Margaret,_Countess_of_Snowdon 0.3333333 0.3333333
## Queen_Victoria 0.3333333 0.3333333
## United_Kingdom 0.0000000 0.0000000
## United_States 0.0000000 0.3333333
## Winston_Churchill 0.3333333 0.3333333
## World_War_I 1.0000000 0.3333333
## World_War_II 0.3333333 1.0000000